Why Is Multiclass Classification Hard?

In classification problems, as the number of classes increases, correctly classifying a new instance into one them is assumed to be more challenging than making same decision in presence fewer classes. The essence problem that using learning algorithm on each boundary individually better several simultaneously. However, why and when it happens still not well-understood today. This work’s main contribution introduce concept heterogeneity boundaries an explanation this phenomenon. Based definition boundaries, we analyze explain differences performance state art approaches solve multi-class classification. We demonstrate performances all approaches, except one-vs-one, decrease. show by encoding knowledge decomposition problem, can obtain results decompositions. benefits increase or decrease time takes train evaluate models. first provide intuitions illustrate effects synthetic datasets simplistic classifier. Then, how real dataset exhibits these principles, also under realistic algorithms. setting, devise method quantify different use decompose problem. significant improvements over state-of-the-art decompositions do take account.